Existing video coding standards and technologies, such as MPEG-4 AVC/H.264, VC1, VP8, and the HEVC/H.265 video-coding standard, have employed block-based methods for coding information. These methods have included intra and inter prediction, as well as transform, quantization, and entropy coding processes. Intra and inter prediction exploit spatio-temporal correlation to compress video data. The transform and quantization processes, on the other hand, have been used to correct errors that may have incurred due to inaccuracies in prediction, given a constraint in bit rate or target quality. The bit rate or target quality has been primarily controlled by adjusting the quantization level for each block. Entropy encoding has further compressed the resulting data given its characteristics.
Although the above processes have resulted in substantial compression of an image or of video data, the inherent block characteristics of the prediction and coding process have resulted in coding artifacts that could be unpleasant and may result in deteriorating the performance of the coding process. Existing techniques introduced in some codecs and standards have attempted to reduce such coding artifacts. Some of these existing techniques applied a “deblocking” filter after reconstructing an image.
Deblocking filters have analyzed a variety of information about a region or block that has been coded and applied filtering strategies to reduce any detected coding artifacts. In codecs such as MPEG-4 AVC, VC1, VP8, and HEVC, the information may include the type of coding mode used for prediction, such as intra or inter, the motion vectors and their differences between adjacent blocks, the presence or absence of residual data, and the characteristics and differences between the samples that are to be filtered. The process is further controlled by adjusting the filtering process given the quantization parameters that were used for the samples currently being filtered. These characteristics were selected in an effort to maximize the detection ability of possible coding artifacts, also referred to as blocking artifacts.
Some codecs included an illumination compensation process, such as weighted prediction, as part of the inter-prediction process to further improve prediction performance. Motion compensated samples were adjusted through a weighting and offsetting process, which is commonly of the form of the below equation (1), instead of being copied directly from another area as the prediction signal:y=w·x(mv)+o  (1)
In this equation, y is the final motion compensated signal, x is the motion compensated signal given a motion vector mv, w is the weighting (scaling) parameter, and o is the offset. Illumination compensation has reduced blocking artifacts in different instances and not just during illumination changes, such as fades, cross-fades, flashes, light source changes, and so on. The codecs also enabled the prediction of similar samples within the same image using bi-prediction, or different samples within the same image using multiple instances of the same reference with different illumination compensation/weighted prediction parameters.
Unfortunately, these existing codecs have not considered differences in illumination compensation parameters during the de-blocking process. For example, in some instances where two adjacent blocks use the same reference but have different illumination compensation parameters, no de-blocking was performed. This caused blocking artifacts to appear across two neighboring blocks from the same reference even though the illumination compensation parameters are different. The blocking artifacts appeared because existing codecs, such as AVC and HEVC, only examine if the actual references used for prediction are the same, and do not consider whether any additional transformation beyond motion compensation has been applied to the reference samples.
FIG. 1 shows an exemplary process 100 of how existing codecs have determined a deblocking filter strength. In box 101, a block boundary between two pixel blocks p and q may be identified. In box 102, a determination may be made as to whether any of the samples of blocks p or q are intra-coded. In box 103, if at least one of the samples is intra-coded, a determination may be made as to whether the identified boundary in block 101 is a macroblock boundary. If the boundary is a macroblock boundary, then in box 104, the block filter strength may be set to a maximum value, such as 4 in this example.
If the boundary is not a macroblock boundary, then the block filter strength may be set to a non-zero value so that deblocking will be performed. For example, the lesser value 3 in box 105 or the lesser value 2 in box 107 may be used in one example, though other values may be used in other embodiments. If none of the samples is intra-coded, then in box 106, a determination may be made as to whether there are any non-zero transform coefficients such as discrete cosine transform (DCT) or discrete sine transform (DST) coefficients in either block p or block q. If there are any non-zero DCT coefficients in either block p or block q, then the block filter strength may be set to a lesser value, such as value 2 in box 107.
If there are not any non-zero DCT coefficients in either block p or bock q, then in box 108 a determination may be made as to whether blocks p and q have different reference pictures or different numbers of reference pictures. If blocks p and q have different reference pictures or different numbers of reference pictures, then the block filter strength may be set to a lesser value, such as value 1 in box 109.
If blocks p and q do not have different reference pictures or different numbers of reference pictures, then in box 110, a determination may be made as to whether a difference between the motion vectors of blocks p and q in either the horizontal direction or the vertical direction is greater than or equal to a threshold. In the example shown in FIG. 1, the threshold is 4, but in other embodiments different thresholds may be used.
If the difference between the motion vectors of blocks p and q in either the horizontal direction or the vertical direction is greater than or equal to the threshold, then the block filter strength may be set to a lesser value, such as value 1 in box 109, which may be the same lesser value that is set when the blocks p and q have different reference pictures or different numbers of references pictures.
If the difference between the motion vectors of blocks p and q in either direction is less than the threshold, then filtering may be skipped and the block filter strength may be set to a zero or least value, such as value 0 in box 111.
As shown in FIG. 1, block filtering may be skipped when two blocks p and q have similar reference pictures and the motion vector difference between the blocks is less than a threshold value even if additional transformations have been applied to one or more reference samples to generate distinct image blocks p and q from one or more similar reference samples. Thus, blocking artifacts may still be present in the outputted images in these instances when filtering is skipped.
There is a need to eliminate blocking artifacts in those instances where additional transformations have been applied to one or more reference samples to generate distinct image blocks from one or more similar reference samples.